Studies on acceptance, evaluation and impact of the Cologne program “Research and Medical Studies”

Introduction: The curricular implementation of events (or programs) for science-related training in human medicine has been on the agenda of the medical faculties since the publication of the Federal-State Working Group [1]. The Medical Faculty of the University of Cologne developed and established a systematic, longitudinal science curriculum together with the start of the model curriculum in human medicine in 2003. Here, we investigate the questions of whether the described (para-) curricular elements are accepted by students and lecturers and how they are evaluated, especially by students. In addition, we investigate whether selected parameters can be used to demonstrate changes in the students' scientific activities. Project description: The program “Research and Medical Studies” (RaMS) consists of several components: these elements of the mandatory curricular (Scientific Projects, SP) and optional components (Research in Medical Studies (RiMS), Research Track (RT), Research Fair Cologne (RFC)) are described here. Results were recorded at various levels: Likert Scale evaluation of the event’s elements were collected as satisfaction parameters from the students Process data on participation in the voluntary events were collected and evaluated as absolute and relational figures (WS 12/13-SS 17). Data on the outcome of the RaMS program were collected: Type of scientific projects in the academic years 2011/12-2014/15), number and type of available projects offered at the RFC (in the years 2011-18) and number of student research funding applications in a comparison of the periods 2010-13 vs. 2014-17). Results: The students’ acceptance of mandatory and paracurricular courses of the RaMS program is pleasingly high, which is not surprising, at least in the case of the voluntary courses. The participation of students in RiMS, RT and RFC is satisfactory for voluntary courses. In the case of the RT, with certified participation of approximately 47% of all registrations (corresponding to 10% of the total cohort), this is comparable to similar programs. It can be shown that the number of experimental science projects has more than doubled over time in parallel with the development of RaMS. The average number of provided projects according to the RFC is 42 (which corresponds to a placement rate of approx. 1:4). The number of successful student applications for a research support grant during the period the measures were implemented has doubled. Discussion and conclusion: The RaMS program shows a route for the implementation of the SP required by the next licensing regulations in medical education, which was initially supported and expanded solitarily, later by further elements (RiMS), also in the sense of a science-based career development (RT, RFC). The student acceptance and the measured success, in the form of successful participation in the Research Track, increased choice of experimental projects, significant increase of submitted as well as approved research grants and the high project placement rate of the Research Fair, encourage the further development of the program, which is indicated in the conclusion.


Introduction
The teaching of scientific competence in medical studies is the subject of an intensive, current academic debate [2], [3], [4], [5]. The Federal-State Working Group has released the "Master Plan for Medical Studies 2020" [1], the implementation of which is likely to establish the teaching of scientific skills as an examination requirement in human medical education. The Flexner Report [6], published more than 100 years ago, already described and criticized the profit-oriented and little scientific education of future medical doctors in America at that time. In Flexner's view, formal-analytical critical thinking, which is the basis of all scientific activity, should be the foundation of medical education and also of all medical practice. The German National Competence-Based Learning Objectives for Medical Education (NKLM) [http:// www.nklm.de], adopted by the Medical Faculty Association (MFT) in 2015, following the CanMEDs model [7], contains and requires the role of a "scholar", science-oriented teaching as well as independent work of a research question for every student of human medicine. Compared with other -divergent -international frameworks for scientific studies in medicine, the role of a scholar in the NKLM thus defines a comparatively stronger methodological orientation [8], [9] as crucial. Thus, the catalogue of learning objectives was accused of being too practiceand school-oriented during its development [10]. In contrast, the German Science Council views the scientifically oriented competencies in the NKLM as a way of firmly integrating the explicit, systematic teaching of scientific competencies in the education of human medicine [11]. Furthermore, the learning goal structure of the NKLM offers the possibility of relatively free design by the faculties and therefore the possibility of creating a "modern" curriculum in every sense, as described by Stern and Papadakis [12]. The German Science Council has published "Recommendations for the Further Development of Medical Education", which, among other things, requires the strengthening of the scientific competence of students [11]. For the latter, "the obligatory acquisition of scientific competence during studies (...) is a necessary prerequisite for the responsible practice of medicine". It has been shown internationally that scientific programs in medical studies can have a positive influence on subsequent research-oriented career development [13], [14], [15]. Based on these recommendations, the question arises for the medical faculties, how it is possible to establish curricular and paracurricular courses in order to integrate the teaching of scientific competences into the already time-consuming medical studies. This thesis aims to answer two research questions in the context of local (para-) curricular implementation: 1. are the described (para-) curricular elements well accepted by students and lecturers and how are they evaluated by students in particular?
2. is it possible to describe a qualitative change in the scientific activities of students on the basis of the selected parameters (types of scientific projects, number of applications in the "Köln Fortune" scholarship program)?

Project description
Mainly curricular elements were designed, implemented and evaluated as a methodological basis. The elements (for an overview see table 1 and figure 1) and the associated evaluation methods are described below. With the exception of the "Scientific Projects" (see below), all courses are open to human and dental medicine students.

Informative Event -Research in Medical Studies (RiMS)
The informative event RiMS is offered to students in the first semester. Students are given an overview of the research landscape in Cologne, information about the Cologne scientific curriculum and the paths to science by present scientists or researching physicians are shown. After this introductory part, students visit research laboratories to gain a first insight into experimental, biomedical fundamental research. In order to evaluate the course, an evaluation sheet with a 6-level Likert scale (liked it -disliked it) was used to assess the individual parts of the course, the overall evaluation of the course by the students and additional characteristics (gender, semester, reason for attendance) (see attachment 1). The data was collected continuously for the courses from the winter semester 2012/13 to the summer semester 2018.

Research Track (RT)
The RT is an optional program that is intended to give students interested in research the opportunity to carry out experimental research projects at an early stage (for a summary see figure 2). The RT program in the first stage of study comprises two lecture series held in English in which scientists from the Faculty of Medicine present their research (mainly from fundamental science areas). The core learning objective is to understand the scientific method of developing hypotheses based on a research question and to test these hypotheses experimentally. Practical teaching elements include visits to research laboratories, the presentation and explanation of large-scale research equipment and techniques, and visits to research facilities of cooperating institutions such as the "German Aerospace Center" or the "German Sport University Cologne". RT participants must complete a final experimental internship of at least 60 hours (this can be accredited as Scientific Project I).
The successful participation in the RT will be certified. The continuation of the RT In the second stage of study is offered as a "lunch seminar". Here, the main focus is finding a suitable doctoral project, not obtaining a second certificate. The time course of the RT curriculum is shown in figure 2.
In both parts of the RT, a student evaluation was carried out after each course, in which student satisfaction was assessed on a 6-level Likert scale (liked it -disliked it). At the end of the semester, an evaluation sheet was distributed, on which items were assessed according to the time of the event (for 1 st stage of studies Wednesdays 6-8 p.m.; for 2 nd stage of studies lunch break 1-2 p.m.), the agreement to English-language lectures and other items (see attachment 2) and the overall assessment of the event by the participants. The proportion of successful RT participants was expressed as a percentage of the number of students who completed the RT completely (i.e. 20 courses were attended and the experimental internship completed) in relation to the number of students who registered for the RT by e-mail after the first lecture of the respective semester.

Research Fair Cologne (RFC)
The RFC is a joint project of the students of the Medical Student Council and the Dean of Studies and Research of the Medical Faculty and has been offered every summer semester since 2013. The DoktaMed of the LMU Munich [16] served as orientation and model for this event and is organized analogous to a scientific poster presentation. The aim of the Research Fair is on the one hand to provide an overview of the currently active research groups of the faculty and on the other hand to enable a direct encounter between students and scientists in order to initiate scientific projects and doctoral theses. In order to evaluate the event, an evaluation sheet with a 5-level Likert scale was developed by the Medical Student Council (liked it -disliked it). This sheet was used to assess the overall evaluation of the event by the students and to ask for additional characteristics (gender, semester, reason for attendance, project found?). The data collection took place between winter semester 2012/13 and summer semester 2018. In order to measure the success of the project placement by the Research Fair, an outcome evaluation was carried out 6 weeks after the Research Fair, during which the participating working groups were interviewed by e-mail and telephone. The following parameters were recorded: Number of project placements, type of projects and whether a publication is planned. For the sake of simplicity, the project types were divided into the following categories: experimental, clinical-statistical and literature work.  The RT consists in both study stages of a lecture series, which is offered over two semesters. The RT in the 1 st study stage includes a mandatory experimental practical internship, which can be accredited as Scientific Project I. In the 2 nd stage of studies, the RT should lead to a medical doctoral thesis, whereby the project outline for the doctoral thesis can be accredited as Scientific Project II. In both stages, the students will be actively motivated to take a research semester off for their doctoral thesis and will be informed about the existing funding possibilities within the framework of Köln Fortune and other scholarship providers.

Scientific Projects I and II (SP I and II)
SP are an integral part of the Cologne model study program and are equivalent to the electives required in the ÄApprO [17]. In each study stage, each student has to carry out an own scientific project of about 60-160 hours (according to the study regulations). The projects range from humanities and social science topics to clinically relevant questions and basic experimental research. Literature research on medical-scientific questions, clinical-statistical projects, qualitative analyses and, of course, experimental internships can be carried out [18]. The proof of performance is an internship report in the style of a scientific publication. In order to analyze the types of scientific projects, the evaluation sheets in the student files of the examination office from the academic years 2011/12 to 2014/15 were evaluated and assigned to the following categories according to the title of the projects: Experimental projects, clinical-statistical projects, literature work, non-assignable projects, scientific work of other origins acknowledged as projects (acknowledged projects). The acceptance of the SP is reflected in the evaluation results of the students' satisfaction with the idea of projects as a whole as well as with the projects themselves (pooled data from 4 semesters between 2008 and 2011 (winter semesters 2008/09 and 2011/12 as well as the summer semesters 2008 and 2009)).
Within the framework of the faculty's internal support program "Köln Fortune", every student has the opportunity to apply for a scholarship for his or her work on research projects. Although information about the possibility of student support was already provided, targeted information for interested students was only started with the introduction of RiMS, RT and RFC. The projects will be evaluated by the "Köln Fortune Research Advisory Council" according to their scientific quality and, if appropriate, approved. The number of approved grants was taken from the respective annual reports of "Köln Fortune".

Results
In order to give an overview of the different measures, the temporal course of the scientifically-oriented paracurricular and curricular information and teaching programs is shown in figure 1. In addition, table 1 contains a short description of each individual program for better comprehensibility. The total number of students per semester is 189 in the pre-clinical stage and between 165 and 175 in the clinical stage. Although the courses are open to students of human medicine and dentistry, we have observed that they are almost exclusively attended by students of human medicine. Therefore we are focusing on these students in this paper.

Research in medical studies (RiMS)
About 120-150 students of each first semester regularly participated in the voluntary informative events RiMS in the studied period, and about 40-70 students in the consecutive laboratory visits. The latter group of students judged the event as highly satisfactory (1.5±0.14 on a 6level Likert scale from "liked it" (1) to "disliked it" (6); mean±SD; 12 semesters from WS 12/13 to SS 18 were considered, see figure 3). In each semester, about 40% male (m) students and 60% female (f) students participated in the informative event RiMS (n=197 male and 303 female students; the evaluation was carried out by the participants who had attended the lecture and the laboratory visits; response rate about 80%).

Research Track (RT)
Of the all the students in the second pre-clinical semester, an average of 21% of each cohort (approx. 40 persons) registered for participation in the program through e-mail; the number of successful participants averaged 47% of all those registered (this corresponds to approx. 10% of each cohort, survey period WS 2012/13-SS 17, see figure   4, point A). In the pre-clinical part of the research track, approximately 35% male students and 65% female students participated in each semester (evaluation data: 69 m, 129 f). The response rate of the evaluation, measured in relation to the number of registered participants of the respective semester, was 52% on average for both stages of study in the considered period.
In the clinical part of the research track, an average of 20 participants registered per semester (corresponding to 12% of the total cohort of approximately 170 students, see figure 4, point B). Satisfaction with the pre-clinical Research Track was evaluated by the students as an average of 1.78±0.19 (mean±SD; pooled evaluation data from WS 12/13-WS 17/18) and in the clinical stage as an average of 1.73±0.14 (on a 6-level Likert scale from "liked it" (1) to "disliked it" (6); mean±SD; pooled evaluation data from SS 14-SS 18; evaluation sheet, see attachment 3). In the pre-clinical stage, English as the language of instruction was rated positively by the participants 1.7±0.4 (on a 6level Likert scale from "liked it" (1) to "disliked it" (6); mean±SD, pooled evaluation data from WS 12/13-WS 17/18, see table 1).

Research Fair
Every year, an estimated 150 -200 students and 100 scientists from an average of 30 different clinics and institutes take part in the Research Fair with approximately 50-70 posters. The evaluation of student satisfaction of the Research Fair resulted in a rating of 2.25±0.21 (on a 5-level Likert scale from "liked it" (1) to "disliked it" (6); mean±SD; pooled evaluation data from 2013-2018, see figure 5, point A). The students rated the item "I got a good overview of the faculty's research" with 2.42±0.16 and the question "I got to know interesting Working Groups" with 2.37±0.18 (on a 5-level Likert scale from "agree" (1) to "disagree" (6); mean±SD; pooled evaluation data from 2015-2018). On average, approximately 48 of the students indicated that they potentially found a project (evaluation data from 2015-2018  figure 5, point B). In 2018, 94 % of the projects offered stated that a publication was planned. The response rate to the outcome evaluation was 67.5%±11.7 on average. Over time, an increase in the number of experimental projects can be shown (based on the titles of the performance records "Scientific Project"). The number of experimental projects as a proportion of all scientific projects

Scientific Projects
With 189 students enrolled in the semester, a total of 411 pooled evaluation forms were submitted for Scientific Project I (pre-clinical stage) (response rate 54%). The students rated the basic idea of conducting SPs as slightly positive with an average of 6.0 points (Likert scale 1-10, SD 2.5), better with regard to the project they had conducted themselves (average 7.0, SD 2.5). There is a moderate mathematical correlation between the ratings (r=0.53; Pearson). The students work in median 60 h (25%-quartile 32 h and 75%-quartile 120 h). Only 6 out of 411 (1.5%) evaluations reported that the students felt "exploited".

Discussion
With the introduction of the model curriculum in human medicine in Cologne, competence-oriented scientific education was integrated into the curriculum as early as 2003 with the implementation of the Scientific Projects [19]. In addition, introductory (RiMS), advanced (RT) and supporting courses (RFC) were implemented in the following years, which are continuously offered and attended by the lecturers and students of the Medical Faculty. This shows that it is principally possible to include student scientific work in the core curriculum of medical studies as a proof of achievement, which is required by the WR and also the MFT (in chapters 6 and 14a of the NKLM). Approximately 380 SP are completed per year in the preclinical stage and approximately 340 SP in the clinical stage. It should not be forgotten that the Cologne model is merely a possible implementation of the requirements of WR and MFT, which is now more longitudinally structured at the Medical Faculty in Cologne under the label "Research and Medical Studies (RaMS)". The model focuses on the support of top-level and broad-based scientific research. SP and RiMS ensure the basic qualification of all students, while RT and RFC particularly target the top group of students. Similarly complex, albeit differently structured programs (e.g. Sideres et al. 2018, [13]) also show a successful science-oriented career development of the participants. The acceptance of the individual optional curricular elements by the students is very positive. The Scientific Projects as a mandatory curricular component still perform positively overall (at least with regard to their own projects). With a median of 60 hours, the time spent on the projects is on average within the time frame provided for this in the study regulations (60-120 hours), although it is at the lower end of the scale. This fits in with the students' statements that only very few students (1.5%) report to have been exploited. Since the competencies required by the NKLM (Chapter 14a, Sections 2 and 3 [http://www.nklm.de]) are already taught in the SP, students who attend the voluntary pre- parative and advanced courses (RiMS, RT, RFC) have the opportunity to knowingly plan their careers in a scientifically guided manner (or to promote young scientists in the sense of the faculty). Only some of the students take advantage of the voluntary elements. The number of participants who complete the Research Track amounts to approx. 10% of the students in a year and is thus comparable to similar programs (MFT Workshop for Teaching Scientific Competence in Medical Studies (Würzburg 2017)). In our opinion, the performance of experimental work represents the entire research process from the formulation of the research question to the conduction of experiments and the evaluation and interpretation of the data. Literature work in the context of Scientific Projects (which are not systematic reviews) does not fulfill this. In this context, the increase in experimental work seems to represent a qualitative improvement in terms of mapping the entire research process. Whether the demonstrated effects on the scientific activities (increase in experimental projects and increased application or success with application-related research grants) are causally related to the RaMS program or not, cannot be proven. However, the close temporal relationship to the newly introduced teaching elements and effects studied suggests that this is the case. At the same time, the placement rate in the context of the RFC of approx. 25% of the participating students shows a pleasingly high efficiency of this measure for the students. Furthermore, these and other elements of the RaMS program obviously also generate direct and lasting added value for the teachers: The consistently high level of participation by the faculty in the RFC, RT and also the SP indicates this as well.
In addition to the structural integration of the program in the examination regulations of the model study program, it should be emphasized that such a comprehensive program can only be maintained with the support of a staff member -in Cologne in the form of a research associate with two student assistants (cf. [20]). In addition to the coordinating tasks (particularly the recruitment of lecturers in RT, but also in contact with the Medical Fac-ulty as the governing body of the RFC), the strategic development of the program is the main focus. Although or perhaps because of the positive experience for the past 15 years with the model curriculum at the Medical Faculty in Cologne in conveying science in medical studies in accordance with the recommendations of the WR and MFT, the next consecutive steps in the further development of the concept are already becoming apparent: • The recently founded Graduate School of Human and Dental Medicine (GSHZ Cologne) guides the qualified paracurricular science program in the first own scientific achievement, the doctorate. • With the faculty's decision to change the academic regulations for the Model Study Program 2017, the Cologne Science Skills Lab (KWiS med ) was founded, whose task is the methodical hands-on preparation of students for training in scientific projects (and doctorates). • The reorganization of teaching in the first study stage, which is expected to start in 2021, will be challenging. This will also involve an even better combination of traditional and new elements in the scientifically-oriented curriculum of the model study program.

Conclusion
Taken together, it becomes clear, particularly through the description of the complex intervention to improve the teaching of science in the study program, that a simple strategy, for example one oriented towards researchbased learning [21], cannot be effective without curricular embedding and guidance [13], [22]. Even if the overall effects of the programs can be shown, proof of their effectiveness in the sense of "clarification studies" [23] must still be provided in future individual cases.